Image Capture Device Having Tilt And/Or Perspective Correction

ABSTRACT

Methods and apparatuses are disclosed to correct for tilt and/or perspective distortion in image capture devices. In some embodiments, the method may include reading an orientation measurement associated with a relative position of an image capture device with respect to an object, determining if the orientation measurement is less than a threshold, and in the event that the orientation measurement is less than the threshold, correcting an image obtained by the image capture device. In some embodiments, the apparatus may include an image sensor, a memory coupled to the image sensor, an orientation measurement device coupled to the image sensor, and a distance measurement device coupled to the image sensor, where the image data may be stored in the memory along with a measurement from the accelerometer and along with a measurement from the distance measurement device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a division of U.S. patent application Ser. No. 12/644,800, byJason Hau-Ping Chen, Brandon Dean Slack, and David I. Simon, entitledImage Capture Device Having Tilt And/Or Perspective Correction, filedDec. 22, 2009, which is incorporated by reference herein in itsentirety.

BACKGROUND OF THE INVENTION

The present invention relates generally to image capture devices inelectronic systems, and more particularly to image capture deviceshaving the ability to correct for tilt and/or perspective distortion.

Electronic devices are ubiquitous in society and can be found ineverything from wristwatches to computers. Many electronic devices nowhave integrated image capture devices, and so users of these electronicdevices now have the ability to take pictures on an impromptu basis. Forexample, in the event that a user does not have a camera in theirpossession but does have a cell phone or other personal media devicethat includes an integrated image capture device, then the user may beable to take a picture instead of foregoing the opportunity to take thepicture altogether. While the ability to take pictures using theseelectronic devices may be advantageous, it is often difficult for theuser to steady these electronic devices and/or keep them level whiletaking the picture. This lack of ability to steady the electronicdevices and/or keep them level while taking the picture often results indistortion in the picture being tilted and/or having a perspective thatis less pleasing to the user.

In fact, tilted pictures and/or pictures with an incorrect perspectivemay also be taken from cameras. For example, a user may not have atripod when taking a picture with a camera and so the user may take apicture at an angle. Regardless of whether a distorted picture isproduced using a camera or an electronic device having an integratedimage capture device, it is often corrected through post-processing.Unfortunately, this post-processing may require sophisticated imageprocessing software and/or a substantial amount of involvement by theuser to correct the distortion.

SUMMARY OF THE INVENTION

Methods and apparatuses are disclosed to correct or compensate for tiltand/or perspective distortion in image capture devices, either in partor in full. In some embodiments, the method may include reading anorientation measurement associated with a relative position of an imagecapture device with respect to an object, determining if the orientationmeasurement is less than a threshold, and, in the event that theorientation measurement is less than the threshold, correcting an imageobtained by the image capture device.

Other embodiments may include an image capture device that has an imagesensor, a memory coupled to the image sensor, an orientation measurementdevice coupled to the image sensor, and a distance measurement devicecoupled to the image sensor. Image data captured by the sensor may bestored in the memory along with a measurement from the accelerometerand/or a measurement from the distance measurement device.

Still other embodiments may take the form of a method of correctingimage distortion including reading a distance measurement from adistance measurement device, where the distance measurement isassociated with a distance between an image capture device and an objectbeing photographed, reading an orientation measurement associated withan image capture device, and correcting an image data representative ofthe object being photographed using the distance measurement and theorientation measurement.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an image capture device capable of correctingdistortion in photographs.

FIG. 1B illustrates a block diagram of the image capture device.

FIG. 2A illustrates a side view of an embodiment of the image capturedevice.

FIG. 2B illustrates a front view of the embodiment shown in FIG. 2A.

FIG. 3A illustrates operations performed for correcting distortion usingorientation data.

FIG. 3B illustrates on-the-fly operations performed for correctingdistortion using orientation data.

FIG. 4A illustrates an image including distortion along the X axis.

FIG. 4B illustrates the image of FIG. 4A with the distortion corrected.

FIG. 4C illustrates the image of FIG. 4A with distortion indicators.

FIG. 5 illustrates operations that may be used to implement thedistortion indicators of FIG. 4C.

FIG. 6 illustrates potential perspective distortion when operating theimage capture device of FIG. 1.

FIG. 7A illustrates an image including perspective distortion.

FIG. 7B illustrates the image of FIG. 7A with the perspective distortioncorrected.

FIG. 7C illustrates the image of FIG. 7A including dynamic crop lines.

The use of the same reference numerals in different drawings indicatessimilar or identical items.

DETAILED DESCRIPTION

Embodiments of electronic devices are disclosed that allow theelectronic device to correct for tilt and/or perspective distortion inphotographs taken with the image capture device. As used herein, theterm “image capture device” is intended to refer to electronic devicesthat have the ability to take still photographs and/or video. Suchelectronic devices may include digital cameras as well as consumerelectronic devices with integrated cameras (e.g., cell phones orpersonal media players). Also, as used herein, the term “photograph” isintended to refer to an image that is selected by the user for storage.The disclosed image capture devices may include accelerometers and/ordistance measurement sensors that record physical orientation data ofthe image capture device with respect to the object being photographed.This orientation data may be used to correct distortion of thephotographs and/or video taken by the image capture device. Theorientation data also may be used in conjunction with distance data tocorrect perspective distortion in the photographs and/or video taken bythe image capture device. In some embodiments, this correction may beperformed by the image capture device on-the-fly as the photographand/or video is being taken. In other embodiments, this correction maybe performed on the photographs and/or video after they are taken. Insuch cases, the orientation and/or distance data may be embedded in theimage data file used to record the photograph and/or video for lateruse. In still other embodiments, the image capture device may utilizethe orientation data and/or distance data to interactively indicate alevel of distortion to the user and allow the user to adjust thephysical orientation of the image capture device to correct thedistortion. For example, in some embodiments, dynamic crop lines or avirtual level may be displayed to the user to indicate the actionnecessary to level the camera.

Although one or more of the embodiments disclosed herein may bedescribed in detail with reference to a particular electronic device,the embodiments should not be interpreted or otherwise used as limitingthe scope of the disclosure, including the claims. In addition, oneskilled in the art will understand that the following description hasbroad application. For example, while embodiments disclosed herein mayfocus on certain portable electronic devices, such as cameras or cellphones, it should be appreciated that the concepts disclosed hereinequally apply to other portable electronic devices that incorporateintegrated cameras. For example, the concepts disclosed herein may beemployed in wristwatches with integrated cameras. In addition, it shouldbe appreciated that the concepts disclosed herein may equally apply tonon-portable electronic devices, such as desktop computers. Furthermore,while embodiments disclosed herein may focus on correcting distortionutilizing accelerometers and/or distance measurement sensors, theconcepts disclosed herein equally apply to other sensors that measurethe physical orientation of the image capture device with respect to theobject being photographed. For example, in some embodiments, the objectbeing photographed and the image capture device may each include globalpositioning system (GPS) devices such that the relative GPS orientationof the object and the image capture device may be recorded along withthe image data. Also, although this disclosure may focus on stillimages, the concepts disclosed herein equally apply to recording movingimages and/or video. Accordingly, the discussion of any embodiment ismeant only to be exemplary and is not intended to suggest that the scopeof the disclosure, including the claims, is limited to theseembodiments.

FIG. 1A illustrates an image capture device 100 capable of correcting,or at least partially compensating for, distortion in photographs. FIG.1B illustrates a block diagram of the image capture device 100. AlthoughFIGS. 1A and 1B illustrate a certain physical layout, it should beappreciated that this is merely for discussion purposes. Referring toFIGS. 1A and 1B, the image capture device 100 may include an aperture110 capable of controlling the amount of light entering the imagecapture device 100 and passing this light along to an image sensor 120through a lens 121. Implementations of the image sensor 120 may varybetween embodiments. For example, in some embodiments, the image sensor120 may be implemented using a complementary metal oxide semiconductorsensor.

The image sensor 120 may be coupled to a processor 130 (as shown in FIG.1B) that controls the general operation of the image capture device 100.In some embodiments, the image sensor 120 is actuated through a switch125, where the switch 125 may be a physical switch on the image capturedevice 100 as shown in FIG. 1A, or alternatively may be a capacitivelycontrolled switch on a display screen 170. In other embodiments, theimage sensor 120 may be actuated by the processor 130 without the switch125, such as with a software interface that may be actuated separatefrom the display screen 170.

In addition to being coupled to the image sensor 120 and the switch 125,the processor 130 may couple to one or more orientation sensors, such asan accelerometer 150 and a distance measurement sensor 155. In someembodiments, the accelerometer 150 may be a micromechanical threedimensional accelerometer, such as the LIS302DL model available fromSTMicroelectronics. Other embodiments may employ gyroscopes, inertialreference sensors, and/or compasses in place of the accelerometer 150 orin conjunction with the accelerometer 150. As the image capture device100 is rotated about any of the X, Y, and/or Z axes the accelerometer150 may report this movement to the processor 130.

The distance measurement sensor 155 may be implemented using an activeauto focus system that includes ultrasonic and/or infrared sensors thatemit sound and/or light respectively. The distance between the imagecapture device 100 and an object 160 being photographed can then bedetermined by measuring the time of flight of delay in either the soundor light reflected from the object 160. In other embodiments, thedistance measurement may be obtained by determining the focal positionof the lens 121—i.e., correlating a physical position of the lens 121 toa distance between the object 160 and the image capture device 100.

As shown in FIG. 1B, the processor 130 may further couple to a memory165 that stores image data optimally, as well as orientation anddistance data, under the direction of the processor 130. A display 170also may couple to the processor 130 to give a user of the image capturedevice 100 an idea of what the image that is being photographed lookedlike. In some embodiments, the user may depress the switch 125 and apotential image of the object 160 may be displayed on the display 170.The image capture device 100 also may include an audible alert device190 that couples to the processor 130 and is capable of generating anaudible alert under the direction of the processor 130. As will bedescribed in greater detail below, this audible alert may be used tocommunicate certain information to the user, such as if a potentialimage includes distortion.

FIGS. 2A and 2B illustrate an embodiment of the image capture device100, specifically a handheld device such as a cell phone or personalmedia device. In some embodiments, the image capture device 100 shown inFIGS. 2A and 2B may be an IPHONE mobile phone or an IPOD TOUCH portablemedia player, both available from Apple Inc. In the embodiments wherethe image capture device 100 is implemented as an IPHONE, then theaudible alert device 190 may be the speaker of the IPHONE and thedisplay 170 may be the screen of the IPHONE.

Regardless of the particular implementation of the image capture device100, during operation, light reflected from the object 160 may betransmitted through the aperture 110 to the image sensor 120. The imagesensor 120 may convert this incident light to image data. When aphotograph is taken by the user, such as by depressing the switch 125,this image data then may be stored by the processor 130 in the memory165 along with orientation data from the accelerometer 150 and/ordistance data from the distance sensor 155. Orientation data generallyrefers to data related to the orientation of the image capture device100 with respect to its surroundings. For example, in some embodiments,the orientation data discussed herein refers to measurements of theEarth's gravitational pull along the X, Y, and Z axes as measured by theaccelerometer 150. In other embodiments, the accelerometer 150 may beused to determine if the image capture device 100 is moving, e.g., in avehicle, and the orientation data may represent the velocity oracceleration of the image capture device 100. Distance data generallyrefers to a distance between the image capture device 100 and the objectbeing photographed. As was alluded to above, the distance data may bethe result of time of AF measurements, a function of the focal positionof the lens 121, or alternatively, may be the result of differencesbetween the GPS coordinates of the image capture device 100 and theobject being photographed.

In some embodiments, the orientation data and/or distance data may bestored into the memory 165 as metadata linked to the image data. Forexample, in some embodiments, this data may be stored in a format thatis compatible with the International Press Telecommunications Council(IPTC) format, such that the orientation and distance data are stored inthe memory 165 as part of a header of the image data file. In otherembodiments, the orientation and distance data may be stored inExchangeable Image File Format (EXIF). For example, an EXIF file may bemodified to include custom tags within the EXIF file that store threeaxis orientation data recorded by the accelerometer 150 and/or thedistance a recorded by the distance sensor 155.

In some embodiments, the processor 130 may utilize the orientation datarecorded by the accelerometer 150 and/or the distance data recorded bythe distance sensor 155 to correct image distortion on the display 170on-the-fly as the photograph is taken. In other embodiments, the imagecapture device 100 may notify the user that image distortion is presentin the image that is about to be photographed.

FIGS. 3A and 3B illustrate two series of operations 200, 205 that may beperformed by the image capture device 100 to correct for distortionusing orientation data. Such distortion may include tilt of the imagedata in the X, Y, and/or Z directions. Operations 200 may apply tocorrecting a photograph after it is taken whereas operations 205 mayapply to correcting an image prior to taking the photograph. Referringfirst to the series of operations 200 in conjunction with FIGS. 1A and1B, in operation 207, a photograph of the object 160 may be taken andrecorded in the memory 165. The stored photograph may include image dataas well as orientation and/or distance data. (The use of distance dataalong with orientation data to correct perspective distortion isdiscussed in greater detail below). In addition to recording the imagedata associated with the photograph during operation 207, orientationdata from the accelerometer 150 may be stored in the memory 165. Theorientation data may be linked with the image data. For example, in someembodiments, the orientation data may be embedded into a header of theimage data file. Continuing the example, the header may be in the IPTCformat. Furthermore, in other embodiments, the orientation data may berecorded with a time stamp that corresponds to a time stamp of the imagedata. For example, the processor 130 may generate a time stamp when theimage sensor 120 obtains an image of the object 160, and this time stampmay be used to index the image data to the orientation as theorientation data is stored in the memory 165. Because the image data aswell as the orientation data are indexed with a time stamp, they may bestored at different locations within the memory 165. This may simplifythe memory management tasks of the processor 130. Note that memory 165may exist locally within the image capture device 100, or alternatively,may exist in a location that is remote to the image capture device 100.For example, the image capture device 100 may send the image datathrough a wireless connection to a remote storage location.

Next, in operation 210, the orientation data may be read by theprocessor 130. For example, in some embodiments, the processor 130 mayread the header data of the ITPC formatted image data to obtain theorientation data. Other embodiments may include the header data beingread by a processor that is external to the image capture device 100.Regardless of where the header data is read, based upon this reading,the physical orientation of the image capture device 100 may bedetermined with respect to the X, Y, and/or Z axes, such as the angle oftilt in the X, Y, and/or Z axes.

In some cases, the user of the image capture device 100 mayintentionally tilt the image capture device 100 with respect to the X,Y, and/or Z axes when photographing the object 160. Thus, the angle oftilt read in operation 210 may represent a deliberate shooting angle.Accordingly, in order to discern deliberate tilt of the image capturedevice 100 from unintentional tilt, the processor 130 may determine ifthe orientation reading is greater than a threshold that is associatedwith deliberate tilt with respect to the X, Y, and/or Z axes, as is donein operation 215. In some embodiments, the threshold may be fivedegrees. Thus, any tilt greater than five degrees may be interpreted bythe image capture device 100 as intentional and not compensated.Furthermore, in some embodiments, the threshold may be programmable bythe user. Also, the threshold may include three independent thresholdsfor the X, Y, and/or Z axes such that the X axis has a differentthreshold than the Y or Z axis and the Y axis has a different thresholdthan the X or Z axis, and so on. Note that the threshold levels may beauto-generated, automatically refined over time by software based uponuser preferences, determined by analyzing a database of similar photos,and/or varied based on inputs from other sensors (e.g., distancemeasurements may indicate more aggressive threshold levels for objectsfurther away).

In the event that the orientation data is greater than the selectedthreshold value, then the tilt may be interpreted by the processor 130as intentional and the photograph may be stored in the memory 165without correction, as shown in operation 220. On the other hand, in theevent that the processor 130 determines that the orientation reading isless than the threshold, then the photograph may be corrected prior tostorage in the memory 165 per operation 225. The correction operation225 may include a variety of operations, such as adjusting thephotograph clockwise and/or counter clockwise to remove theunintentional tilt prior to storage in the memory 165. Since thethreshold comparison in operation 215 may include different thresholdsin multiple dimensions, the ultimate determination as to whether theimage capture device 100 is deliberately tilted (such that no correctionto the photograph is made before storage in the memory 165) may varybetween embodiments. For example, in some embodiments, if theorientation reading indicates one or more of the three dimensions, thenthe photograph may be corrected (per operation 225) in the dimensionthat exceeds the threshold value. In other embodiments, the photographmay not be corrected (per operation 225) unless the orientation readingindicates that two of the three dimensions are greater than the theirrespective thresholds. In still other embodiments, the photograph maynot be corrected (per operation 225) unless the orientation readingindicates that all three of the dimensions are greater than the theirrespective thresholds. In yet other embodiments, a transformationcorrection filter may be computed regardless of orientation thresholds,where a limit to the amount of transformation may be calculated and usedinstead of the orientation thresholds.

In at least one embodiment, the correction operation 225 may includeapproximating an angle of straight edges in the captured image. If thestraight edges become very close to being vertical after theaccelerometer data is applied to straighten the captured image, then theentire captured image may be made substantially vertical by applying thechanges made to the straight edges to the rest of the captured image.Thus, in these embodiments, the thresholds may be used to determine howclose the straight edges are to being vertical.

In addition to correcting the photograph prior to storing it in thememory 165, the photograph may be corrected on-the-fly when displayingan image of the potential photograph to the user. This is illustrated inthe operations 205 shown in FIG. 3B. Referring now to the operations 205in conjunction with FIGS. 1A and 1B, an image of the potentialphotograph may be displayed on the display 170 in operation 230. Thismay occur as a result of the user depressing the switch 125 to indicatethat the user is about to take a photograph. Orientation data from theaccelerometer 150 may be read by the processor 130 during operation 240and used to determine if the image of the potential photograph includesdistortion. For example, the reading taken during operation 240 may beused to determine if the image displayed to the user on the display 170(i.e., the potential photograph) includes distortion, or alternatively,if the user has deliberately tilted the image capture device 100. Thisis shown in operation 250. As was the case with operation 215, operation250 may include determining whether the image capture device 100 hasbeen deliberately tilted by comparing the orientation data reading fromoperation 240 with one or more threshold values. In the event that theorientation data read from the accelerometer 150 is greater than thethreshold, then the processor 130 may interpret this as deliberate tiltby the user and forego correction of the image displayed to the user onthe display 170. This is shown in operation 260. In the event that theorientation data read during the operation 240 is less than thethreshold value, then the processor 130 may interpret this imagedistortion as deliberate and correction may be performed on the imageprior to taking the photograph per operation 270 and the corrected imagemay be displayed to the user per operation 280. In this manner, the usercould determine if the correction was adequate prior to taking thephotograph.

FIGS. 4A and 4B respectively illustrate distortion and on-the-flycorrection of an image distorted in the X axis. Although FIGS. 4A and 4Bfocus on distortion along the X axis of the image for the sake ofdiscussion, this discussion equally applies to distortion along Y and/orZ axes as well. Referring now to FIGS. 4A and 4B, FIG. 4A illustrates animage of the United States Capitol Building that may be displayed on thedisplay 170. As can be appreciated from inspection of FIG. 4A, the imageof the Capitol Building is tilted along the X axis. For the sake ofdiscussion, it is assumed that the image shown in FIG. 4A is tilted lessthan the threshold amount indicated in operation 250—i.e., the tilt isunintentional. Because of this, the image displayed on the display 170may be corrected on-the-fly per operations 205. FIG. 4B illustrates thissame image in corrected form where the image is substantially free of Xaxis distortion per operation 280. Now, when the user takes thephotograph, the image data stored in the memory 165 may be substantiallyfree of distortion. In these embodiments, since the orientation data hasbeen used to correct the image data on-the-fly, then the accelerometerdata optionally may be stored in the memory 165, or in order to conservespace in the memory, the orientation data may be discarded.

In the embodiments where correction on-the-fly is performed, the imagecapture device 100 may alert the user visually, audibly, physicallythrough vibration feedback, and/or haptics that the correction hasoccurred. For example, in some embodiments, when the image capturedevice 100 has performed on-the-fly correction, the image capture device100 may indicate this to the user by actuating the audible alert device190. In other embodiments, such as when the image capture device 100 isa cell phone, the phone may alert the user through vibration that acorrection has occurred. In still other embodiments, the image capturedevice 100 may indicate that on-the-fly correction has been performedvisually to the user by displaying an on-the-fly distortion correctionicon (not specifically shown) on the display 170. In yet otherembodiments, instead of correcting the image (operation 270) anddisplaying the corrected image (operations 280) the image capture device100 may display distortion indicators 305 on the originally displayedimage such that the user can gauge the amount of cropping that may takeplace prior to allowing the image capture device 100 to perform theon-the-fly correction and storing the photograph in the memory 165. FIG.4C illustrates the distorted image of the Capitol Building from FIG. 4Awhere the distortion indicators 305 have been imposed on the imagedisplayed on the display 170. The distortion indicators 305 may becalculated by the processor 130 such that they correspond with a desiredorientation along the X, Y, and/or Z axes. For example, referring toFIG. 4C in conjunction with FIG. 1A, the distortion indicators 305 areshown as orthogonal to the plane of the image capture device 100 definedby the X and Y axes.

FIG. 5 illustrates the operations 400 that may be used to implement thedistortion indicators 305 shown in FIG. 4C. As was the case for theoperations 205, the operations 400 may begin by displaying the image tothe user in operation 405, reading orientation data in operation 410 anddetermining whether the orientation data is greater than the thresholdin operation 420. In the event that the orientation data indicates thatthe distortion is unintentional, i.e., tilt is less than the threshold,then the processor 130 may display the distortion indicators 305 peroperation 430. In some embodiments, this may occur as a resultdepression of the switch 125 to indicate the user desires to take aphotograph of the image on the display 170 (during operation 405), andtherefore, the user may have the opportunity to manually correct theimage by tilting the image capture device 100 to align the image withthe distortion indicators 305 shown on the display 170 during operation430. In the event that operation 420 indicates that the orientation isgreater than the threshold (e.g., the tilt is deliberate), then thedistortion indicators 305 may be omitted from the display 170 peroperation 430.

In addition to correcting for image distortion in the X, Y, and/or Zaxes, the orientation data as measured by the accelerometer 150 may beused in conjunction with distance data to correct perspective distortionpresent in the image of the object 160. The term “perspectivedistortion” generally refers to a warping of the object 160 that stemsfrom the image sensor 120 and the object 160 being at angles withrespect to each other. FIG. 6 illustrates potential perspectivedistortion when operating the image capture device 100 shown in FIG. 1.Referring to FIG. 7, the object 160 is in a substantially non-verticalposition with respect to the ground and/or horizon, indicated by θ₁. Asa result of this relative non-vertical positioning, the image presentedto the image sensor 120 may be warped or skewed and the photograph ofthis image will have perspective distortion. For example, FIG. 7Aillustrates an image of Big Ben that may be displayed on the display170, including perspective distortion.

In some embodiments, the distance measurement sensor 155 may provide oneor more distance measurements to be used in conjunction with theorientation data from the accelerometer 150 in order to correct forperspective distortion. For example, in some embodiments, the distancemeasurement sensor 155 may measure the distance d₁ of a vector that isorthogonal to the image sensor 120 and extends between the image sensor120 and the object 160. Additionally, the distance measurement sensor155 may measure the distance d₂ of a vector that is parallel to theground and extends between the image sensor 120 and the object 160.Furthermore, the accelerometer 150 may measure the angle of the imagesensor 120 with respect to the ground θ₂. Based upon the distancemeasurements d₁ and d₂ as well as the angle θ₂, the angle of the object160 with respect to the horizon θ₁ may be determined throughtrigonometric operations. By calculating the angle θ₁ with the processor130, perspective distortion may be corrected for using a perspectivetransformation operation on-the-fly prior to storing the image data inthe memory 165. As mentioned above, such on-the-fly correction mayconserve space in the memory 165. FIG. 7B illustrates the image of FIG.7A processed with a perspective distortion transformation. In otherembodiments, the distance measurements d₁ and d₂ as well as the angle θ₂may be stored in the header of the image data so that the perspectivetransformation may be applied by calculating the angle θ₁ at a latertime.

As can be appreciated from comparing FIGS. 7A and 7B, a portion of theimage data was cropped out of FIG. 7A to preserve the aspect ratio ofthe original image when correcting for perspective distortion.Similarly, as can be appreciated from comparing FIGS. 4A and 4B, aportion of the image data was cropped out when correcting for tiltdistortion. Because image data is to be cropped out of the image whencorrecting for tilt or perspective distortion, some embodiments mayindicate the portion that is to be cropped out to the user on thedisplay 170 using dynamic crop lines. FIG. 7C illustrates dynamic croplines 505 imposed on the image illustrated in FIG. 7A. The dynamic croplines 310 may aid the user in framing the object to be photographed suchthat user desired details are preserved after tilt and/or perspectivedistortion correction.

We claim:
 1. A method of compensating for image distortion comprisingthe acts of: obtaining an orientation measurement of an image capturedevice during an image display operation wherein a displayed imagecomprises data representative of a scene. identifying one or morestraight edges in the data corresponding to straight edges in the scene;determining a difference based, at least in part, on the measuredorientation of the image capture device and an orientation of at leastone of the one or more straight edges in the data; modifying the datawhen the difference is less than a threshold value; and displaying themodified data.
 2. The method of claim 1, wherein obtaining theorientation measurement of the image capture device comprises obtainingthe orientation measurement of the image capture device along each ofone or more axes relative to the image capture device.
 3. The method ofclaim 2, wherein determining the difference comprises determining thedifference based, at least in part, on the measured orientation of theimage capture device along a first axis of the one or more axes and anorientation of at least one of the one or more straight edges in thedata along the first axis.
 4. The method of claim 2, wherein modifyingthe data comprises modifying the data so that the orientation of the atleast one of the one or more straight edges in the data is aligned withthe measured orientation of the image capture device.
 5. The method ofclaim 2, wherein modifying the data comprises modifying the image dataso that the orientation of the one or more straight edges in the data isaligned near but not exactly with the measured orientation of the imagecapture device.
 6. The method of claim 2, wherein determining thedifference comprises: determining a first angular difference between anorientation of a first of the one or more straight edges in the data andthe measured orientation of the image capture device along a first ofthe one or more axes; and determining a second angular differencebetween an orientation of the first straight edge and the measuredorientation of the image capture device along a second of the one ormore axes.
 7. The method of claim 6, wherein modifying the datacomprises: modifying the data so that the orientation of the firststraight edge is aligned with the measured orientation of the imagecapture device along the first axis based on, at least in part, thefirst angular difference; and modifying the data so that the orientationof the first straight edge is aligned with the measured orientation ofthe image capture device along the second axis based on, at least inpart, the second angular difference.
 8. The method of claim 1, whereinmodifying the data comprises displaying a distortion indication on adisplay, wherein the distortion indication is related to the measuredorientation of the image capture device.
 9. A non-transitory programstorage device, readable by a processor and comprising instructionsstored thereon to cause one or more processors to: obtain an orientationmeasurement of an image capture device during an image displayoperation, wherein a displayed image comprises data representative of ascene; identify one or more straight edges in the data corresponding tostraight edges in the scene; determine a difference based, at least inpart, on the measured orientation of the image capture device and anorientation of at least one of the one or more straight edges in thedata; modify the data when the difference is less than a thresholdvalue; and display the modified data.
 10. The non-transitory programstorage device of claim 9, wherein the instructions to cause the one ormore processors to obtain the orientation measurement of the imagecapture device comprise instructions to cause the one or more processorsto obtain the orientation measurement of the image capture device alongeach of one or more axes relative to the image capture device.
 11. Thenon-transitory program storage device of claim 10, wherein theinstructions to cause the one or more processors to determine thedifference comprise instructions to cause the one or more processors todetermine the difference based, at least in part, on the measuredorientation of the image capture device along a first axis of the one ormore axes and an orientation of a first of the one or more straightedges in the image data along the first axis.
 12. The non-transitoryprogram storage device of claim 10, wherein the instructions to causethe one or more processors to modify the data comprise instructions tocause the one or more processors to modify the data so that theorientation of the at least one of the one or more straight edges in thedata is aligned with the measured orientation of the image capturedevice.
 13. The non-transitory program storage device of claim 10,wherein the instructions to cause the one or more processors to modifythe data comprise instructions to cause the one or more processors tomodify the data so that the orientation of the at least one of the oneor more straight edges in the data is aligned near but not exactly withthe measured orientation of the image capture device.
 14. Thenon-transitory program storage device of claim 10, wherein theinstructions to cause the one or more processors to determine thedifference comprise instructions to cause the one or more processors to:determine a first angular difference between an orientation of a firstof the one or more straight edges in the data and the measuredorientation of the image capture device along a first of the one or moreaxes; and determine a second angular difference between an orientationof the first straight edge and the measured orientation of the imagecapture device along a second of the one or more axes.
 15. Thenon-transitory program storage device of claim 14, wherein theinstructions to cause the one or more processors to modify the datacomprise instructions to cause the one or more processors to: modify thedata so that the orientation of the first straight edge is aligned withthe measured orientation of the image capture device along the firstaxis based on, at least in part, the first angular difference; andmodify the data so that the orientation of the first straight edge isaligned with the measured orientation of the image capture device alongthe second axis based on, at least in part, the second angulardifference.
 16. The non-transitory program storage device of claim 9,wherein the instructions to cause the one or more processors to modifythe data comprise instructions to cause the one or more processors to:display the data and a distortion indication on a display, wherein thedistortion indication is related to the measured orientation of theimage capture device.
 17. An image capture device, comprising: an imagesensor; a lens assembly configured to focus light from a scene onto theimage sensor; a sensor coupled to the image capture device; a displayunit; a memory coupled to the image sensor and the display unit; and oneor more processors coupled to the sensor, the display unit and thememory and programmed to execute instructions stored in the memory tocause the one or more processors to display data representative of thescene on the display unit, obtain a signal from the sensorrepresentative of an orientation measurement of the image capture deviceduring display of the image data on the display unit, wherein theorientation measurement comprises an orientation measurement along eachof one or more axes relative to the image capture device, identify oneor more straight edges in the data corresponding to straight edges inthe scene, determine a difference based, at least in part, on theorientation measurement and an orientation of at least one of the one ormore straight edges in the data, modify the data when the difference isless than a threshold value, and display the modified data on thedisplay unit.
 18. The image capture device of claim 17, wherein theinstructions to cause the one or more processors to determine thedifference comprise instructions to cause the one or more processors todetermine the difference based, at least in part, on the orientationmeasurement of the image capture device along a first axis of the one ormore axes and an orientation of at least one of the one or more straightedges in the image data along the first axis.
 19. The image capturedevice of claim 17, wherein the instructions to cause the one or moreprocessors to modify the data comprise instructions to cause the one ormore processors to modify the data so that the orientation of a firstone of the one or more straight edges in the data is aligned with theorientation measurement along a first of the one or more axes relativeto the image capture device.
 20. The image capture device of claim 17,wherein the instructions to cause the one or more processors to displaythe modified data further comprise instructions to cause the one or moreprocessors to display a distortion indicator on the display unit,wherein the distortion indication is related to the orientationmeasurement of the image capture device.